Papers with confidence quality
When High Accuracy Hides Poor Calibration: Rethinking Confidence Evaluation in Transformer-Based Text Classification with Balanced Brier Score (2026.acl-long)
Copied to clipboard
Guilherme Fonseca, Gabriel Prenassi, Washington Cunha, Leonardo Chaves Dutra da Rocha, Marcos André Gonçalves
| Challenge: | Existing evidence for TC under fine-tuning is limited. |
| Approach: | They propose a calibration method that balances the contribution of correct and incorrect predictions within confidence bins. |
| Outcome: | The proposed calibration measures show that the models are overconfident even when miscalibrated . the proposed calibration methods challenge calibration assessment practices and provide a more reliable alternative for evaluating confidence quality in Transformer-based TC. |